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REPORT:  2/1/83  TO  1/31/84 

I.  OBJECTIVES 


FQ8671-8300517 


The  overall  objective  of  the  project  is  further  development  of  a 
model  (Arend,  1973)  describing  the  role  of  retinal  image  motion 
in  the  human  visual  system's  processing  of  edge  information  in 
patterns.  Several  lines  of  investigation  were  proposed, 
including: 

A.  Eye  Movements  and  Stabilized  Retinal  Images 

Recent  experiments  using  grating  patterns  held  stationary  on  the 
retina  have  raised  questions  concerning  the  role  of  retinal  image 
motion  in  spatial  pattern  vision.  The  fact  that  high  contrast 
gratings  remain  visible  when  stabilized  with  the  SRI  Double 
Purkinje  Image  Eyetracker  (Kelly,  1979)  has  been  interpreted  as 
meaning  that  the  human  visual  system  can  process  pattern 
information  in  the  absence  of  temporal  change  of  the  retinal 
image,  contradicting  previous  views.  This  issue  is  a  critical 
part  of  the  most  basic  understanding  of  human  pattern  vision.  A 
major  goal  of  the  first  year  of  the  project  was  examination  of  PI 
tracker  grating  stabilization  and  Kelly's  interpretation  of  his 
data. 

B.  Suprathreshold  Contrast  Matching  and  Temporal  Modulation 

The  model  proposes  that  all  pattern  information  is  derived  from 
temporal  changes  of  the  retinal  image.  Useful  pattern  perception 
requires  that  the  visual  system  take  the  source  of  the  temporal 
change  (eye  rotation,  object  movement,  fluctuation  of  external 
illumination)  into  account  since  the  different  sources  produce 
different  patterns  and  amplitudes  of  temporal  change  for  the  same 
external  objects.  The  compensation  mechanism  has  consequences 
for  the  relationship  between  threshold  and  suprathreshold  grating 
perception.  A  second  goal  of  the  first  year  was  completion  of  a 
study  of  suprathreshold  contrast  matching  of  temporally  modulated 
sinusoidal  gratings  which  examines  those  consequences. 

C.  Spatial  Gradients  and  Integration  over  Edge  Information 

A  large  component  of  the  project  is  investigation  of  how 
information  from  sharp  and  shallow  spatial  gradients  of  intensity 
and  color  gets  put  together  into  a  stable  spatial  representation 
of  lightness  and  color.  The  first  year  goals  for  this  portion  of 
the  project  were  selection,  purchase  and  installation  of  a 
display  system  consisting  of  minicomputer,  image  processor  and 
color  video  monitor. 
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Substantial  progress  was  made  in  evaluating  Purkinje  Image 
Eyetracker  stabilization  of  gratings.  The  principal  conclusions 
are: 

1)  Most  of  the  gross  form  of  Kelly's  grating  stabilization 
data  replicates  in  our  lab  using  a  similar  eyetracker, 
different  subjects  and  improved  methods. 

2)  Calculations  from  unstabilized  grating  contrast 
sensitivities  show  that  the  P.I.  tracker  (and,  in  fact,  no 
existent  eyetracker)  can  eliminate  all  meaningful  retinal 
image  motion  and  therefore 

a)  Residual  visibility  of  stabilized  objects  does  not 
necessarily  imply  that  the  visual  system  contains  mechanisms 
capable  of  responding  to  temporally  unchanging  inputs  and 

b)  Only  subjective  pattern  fading  and  disappearance  can  be 
relied  upon  as  a  criterion  of  stabilization  adequacy,  and 
then  only  with  qualification. 

In  light  of  current  evidence,  therefore,  it  is  likely  that  all 
pattern  information  is  derived  from  retinal  temporal  changes,  and 
some  model  describing  the  transfer  from  temporal  information  to 
spatial  pattern  is  required. 

1)  Stabi 1 i zed  Contrast  Sensitivities.  The  first  portion  of  the 
year  was  devoted  to  setting  up  optics  and  electronics  to 
stabilize  gratings  on  the  retina  with  a  generation  IV  SRI 
Purkinje  Image  Eyetracker.  A  high  luminance  monitor  (Tektronix 
608)  was  purchased  and  electronics  were  designed  and  constructed 
to  display  vertical  cosine  luminance  gratings.  The  horizontal 
analog  eye  position  voltage  from  the  tracker  was  sampled  and 
digitized  140  times  per  second,  at  the  beginning  of  each  display 
frame,  and  the  phase  of  the  grating  was  offset  to  produce 
constant  phase  on  the  retina. 

For  the  initial  work  conditions  similar  to  Kelly's  were  used 
since  we  were  attempting  to  replicate  his  results.  A  few 
procedures  were  slightly  improved  to  increase  control.  The 
temporal  rate  of  change  of  contrast  in  the  adjustment  procedure 
was  constant,  under  computer  control.  In  Kelly's  experiments  the 
subject  initially  viewed  the  pattern  at  an  unspecified  high 
contrast  for  an  unspecified  duration  in  order  to  rapidly  build  up 
the  afterimage.  This  procedure  gave  highly  variable  results  in 
our  early  work  so  we  held  the  initial  contrast  constant  at  0.6 
for  20  seconds. 

In  fig.  1  are  mean  log  sensitivities  for  stabilized  gratings 
with  superimposed  drift  velocities  of  0,  0.012,  0.15,  and  3 

degarc/s.  The  general  magnitude  of  the  effect  of  pattern 
velocity  and  the  overall  shapes  of  our  curves  agree  with  Kelly's 
( Ke 1 1 y , 1 9 7 9 ) .  Under  these  stabilization  conditions  the 
motionless  grating  threshold  was  elevated  by  a  factor  of  50  over 
the  0.15  deg/s  threshold  at  1  c/deg,  decreasing  monotonically  to 
a  factor  of  seven  at  8  c/deg.  Imposing  a  drift  velocity  of  only 
0.12  deg/sec  (roughly  twice  the  foveal  intercone  distance  per 
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second  of  time)  increased  sensitivity  by  as  much  as  0.75  log 
units. 

The  fact  that  the  zero-velocity  log  sensitivies  do  not  drop 
to  zero  has  been  interpreted  by  Kelly  and  others  to  mean  that  the 
visual  system  is  capable  of  detecting  temporally  unchanging 
gratings  of  sufficiently  high  contrast.  This  interpretation  is 
very  attractive  since  it  greatly  simplifies  the  early  visual 
encoding  of  pattern.  It  would  provide  pattern  vision  modellers 
some  (though  still  inadequate)  justification  for  ignoring  eye  and 
head  movements.  It  holds,  however,  only  if  one  can  believe  that 
all  important  temporal  change  has  been  eliminated  by  the 
stabilization.  Some  of  Kelly's  data  suggest  that  extremely  small 
temporal  changes  might  be  psychophysically  important,  so  one  goal 
of  the  first  year  was  development  of  a  means  of  quantitatively 
evaluating  the  importance  of  small  residual  retinal  motion  of  the 
"stabilized"  gratings. 

I  found  that  the  importance  of  small  oscillations  can  be 
evaluated  by  a  relatively  simple  calculation.  A  grating 
sinusoidally  oscillating  on  the  retina  over  a  distance  which  is 
small  relative  to  one  grating  period  is  approximately  the  sum  of 
two  perfectly  stabilized  gratings.  One  is  temporally-constant 
and  high-contrast;  the  other  is  temporally  counterphase  modulated 
and  of  low  contrast: 

L(x,t)  »  Lm  [1  +  Cssin  2  7T  fx  + 

(Cg2  7T  f  V  )  (cos  2f  V  t)  (cos  2  -TY  fx)  (1) 

where  L(x,t)  is  the  distribution  in  space  and  time,  LM  is  the 
mean  luminance,  C  is  the  grating  contrast,  and  and  \>  are 

the  amplitude  and  temporal  frequency  of  the  grating's  oscillatory 
movements . 

The  second  term  of  equation  (1)  is  a  counterphase  modulated 
grating  of  contrast  Cs2  f  Y  .  If  we  assume: 

1)  as  noted  above,  that  Y  is  small  relative  to  one  grating 
period  and 

2)  the  presence  of  the  high-contrast  stationary  component 
is  neither  directly  detectable  nor  indirectly  affects 
detection  by  altering  the  sensitivity  to  the  temporally 
modulated  component 

then  equation  1  provides  a  link  between  measurable  sensitivity  to 
counterphase-modulated  gratings  and  sensitivity  to  sinusoidally 
oscillating  gratings,  i.e., 

C  V  ,f  -  Cs2  7T  f  t  (2) 

where  C  ^  f  is  the  measured  threshold  contrast  for  a 
counter  phase'-modula  ted ,  stabilized  grating  of  temporal  and 
spatial  frequencies  of  V  and  f,  respectively1  and  the  other 
quantitites  are  as  in  equation  1.  From  this  relation  we  can 

1The  measurement  requires  that  the  counterphase  modulation  be 
greater  than  temporal  modulation  due  to  inadequate  stabilization. 
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calculate,  for  example,  the  amplitude,  Y  ,  of  4-Hz  motion  of  a  2 
c/deg,  100%  contrast  grating  which  would  produce  just-detectable 
temporal  change  on  the  retina.  In  stabilized  counterphase 
grating  experiments  Ca  o  is  observed  to  be  about  0.014. 
Therefore,  ' 

=  C  ^  ,f/Cs2  7T  f  =  0.014/4  =  0.0011  deg  amplitude, 

or  8  secarc  pk-pk.  Since  no  existing  eyetracker  can  avoid  such 
small  movements  a  100%  contrast,  2  c/deg  grating  cannot  be  made 
to  disappear  by  stabilization  alone;  the  visual  system  is 
remarkably  sensitive  to  retinal  movement  of  such  spatiotemporal 
patterns. 

While  the  conclusions  are  robust  when  the  conditions  are  met 
(i.e.,  the  math  is  straightforward)  we  decided  to  collect  data 
allowing  direct  comparison  of  thresholds  for  wiggling  and 
counterphasing  gratings. 

In  fig.  2  are  plotted  temporal  CSFs  for  two  conditions.  The 
points  with  right-hand  error  bars  are  mean  log  sensitivities  for 
1  c/deg  stabilized  gratings,  counterphase  modulated  at  the 
abscissa  frequencies.  The  sensitivities  in  the  other  curve  were 
calculated  (using  eq.  2)  from  thresholds  for  stabilized  1  c/deg 
gratings  moved  sinusoidally  at  the  abscissa  frequencies  over  a 
pk-pk  distance  of  3.33  minarc.  The  agreement  is  excellent 
through  4.4  Hz.  The  second  assumption  underlying  eq.  2  is 
obviously  suspect  with  respect  to  the  breakdown  at  higher 
temporal  frequencies,  but  a  detailed  explanation  is  not  yet 
clear . 

In  fig.  3  are  comparisons  of  thresholds  for  gratings 
drifting  or  counterphase  flickering  at  4.4  Hz.  For  unstabilized 
gratings  the  ratio  of  the  thresholds  decreases  with  increasing 
spatial  frequency  (e.g.,  Watson,  et  al,  1980).  We  found,  as 
Kelly  did,  that  the  ratio  of  thresholds  for  the  two  modulation 
conditions  was  independent  of  spatial  frequency  for  stabilized 
gratings.  However,  our  ratio  was  closer  to  that  found  by  Watson, 
et  al  for  unstabilized  low  spatial  frequency  gratings  than  to 
Kelly's  ratio  of  two.  In  unstabilized  viewing  the  observed  ratio 
decreased  as  spatial  frequency  was  increased.  Comparison  of  the 
stabilized  and  unstabilized  curves  showed  that  retinal  image 
motion  due  to  eye  movements  enhances  detection  of  high  spatial 
frequency  gratings  for  both  temporal  modulation  types,  but  more 
for  counterphase  flicker  than  for  drift. 

We  have  recently  collected  some  data  in  an  initial  effort  to 
tease  apart  several  adaptation  processes  contributing  to  the 
threshold  elevations  in  fig.  1.  In  fig.  4  are  spatial  CSF's  for 
0.15  °/s  drifting  gratings  (included  because  it  is  similar  to  the 
unstabilized,  free-viewing  spatial  CSF)  and  for  a  condition  with 
adaptation  and  criterion  identical  to  the  0  °/s  curve  of  fig.  1, 
but  with  voluntary  fixation  (subject  attempting  to  suppress 
saccadic  eye  movements)  rather  than  tracker  stabilization. 
Comparison  of  these  curves  shows  that  the  adaptation  procedures 
and  voluntary  limitation  of  eye  movements  account  for  a 
substantial  portion  of  the  threshold  elevation  of  fig.  1. 
Nevertheless,  the  4  c/deg  point  of  the  0  °/s  curve,  fig.  1,  is 
0.9  log  units  below  the  corresponding  voluntary  fixation  point  of 


Figure  2.  Mean  log  sensitivities  for  1  c/deg  gratings  temporally 
modulated  by  two  methods.  Right-facing  error  bars:  stabilized, 
counterphase-flicker.  Left-facing  error  bars:  stabilized, 
moving  sinusoidally,  pk-pk  movement  =  3.33  minarc,  sensitivitity 
calculated  from  equation  2.  S:  LA. 


Figure  3.  Mean  log  sensitivities  for  drifting  and  flickering 
gratings.  Circles:  4.4  Hz  drift.  Squares:  4.4  Hz  counterphase 
flicker.  Solid  line:  stabilized.  Dashed  line:  unstabilized. 
S:  LA. 


Figure  4.  Mean  log  sensitivities.  Bottom  curve:  zero-velocity 
stabilized  curve  from  fig.  1.  Middle  curve:  unstabilized, 
voluntary-fixation,  with  adaptation  and  criterion  identical  to 
bottom  curve.  Top  curve:  0.15  deg/s  curve  from  fig.  1. 


Figure  5.  Mean  log  sensitivities.  Bottom  curve:  zero-velocity 
stabilized  curve  from  fig.  1.  Middle  curve:  stabilized  curve 
from  ascending  method  of  limits.  Top  curve:  unstabilized  curve 
from  ascending  method  of  limits. 
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fig.  4,  showing  a  very  substantial  contribution  of  eyetracker 
stabilization. 

In  fig.  5  are  curves  collected  using  adaptation  and 
criterion  procedures  similar  to  those  designed  by  Burbeck  and 
Kelly  (1984).  Their  procedures  were  intended  to  minimize 
duration  of  exposure  to  suprathreshold  gratings  and  therefore 
hopefully  reduce  some  of  the  contributing  adaptive  processes  more 
than  others.  The  bottom  curve  is  the  zero-velocity  curve  from 
fig.  1,  included  for  comparison  purposes.  The  upper  curve  is  for 
unstabilized  viewing  with  the  subject  moving  the  eyes  to 
subjectively  maximize  sensitivity.  The  middle  curve  is  for 
tracker  stabilization.  For  both  of  the  upper  curves  the  plotted 
points  are  the  mean  log  end  points  of  five  ascending  method  of 
limits  runs.  For  each  run  the  grating  contrast  was  set  well 
below  the  threshold,  and  after  20  s  a  tone  indicated  the 
beginning  of  a  5  s  test  period.  The  contrast  was  increased  by 
0.05  log  units  at  the  end  of  the  test  period  and  the  cycle 
repeated  until  the  subject  reported  the  grating  visible 
throughtout  the  5  s  test  period. 

Under  these  conditions  Burbeck  and  Kelly  (1984)  found  much 
less  threshold  elevation  than  in  Kelly's  original  experiments, 
with  no  significant  threshold  elevation  due  to  stabilization  at 
frequencies  above  2  c/deg.  We  found  that  stabilization  under 
these  conditions  still  produced  substantial  threshold  elevation, 
even  at  8  c/deg.  While  these  data  must  be  replicated  with  a 
second  subject,  it  appears  that  less  of  the  elevations  in  fig.  1 
may  be  attributed  to  nonlocal  grating  adaptation  thanindicated  by 
Burbeck  and  Kelly's  data.  This  is  the  first  substantial 
disagreement  of  data  from  the  same  experiment  in  the  two 
laboratories  (but  not,  obviously,  of  interpretation). 


B.  Suprathreshold  Contrast  Matching  and  Temporal  Modulation 


The  shape  of  the  spatial  contrast  sensitivity  function 
depends  strongly  on  the  temporal  modulation  of  the  gratings.  The 
model  attributes  much  of  that  dependence  to  differences  between 
the  patterns  of  temporal  change  produced  by  retinal  image  motion 
versus  temporal  modulation  of  the  viewed  pattern.  Approximately 
veridical  contrast  perception  requires  that  the  source  of 
temporal  change  be  taken  into  account  at  high  pattern  contrasts 
to  offset  different  efficiencies  of  the  various  ways  a  particular 
pattern  may  be  temporally  modulated.  In  other  words,  the 
substantial  differences  between  contrast  thresholds  for  various 
spatial  and  temporal  frequencies  should  be  reduced  or  eliminated 
at  high  physical  grating  contrasts. 

In  pilot  work  I  found  preliminary  evidence  that  temporal 
modulation  had  a  much  smaller  effect  on  the  shape  of 
suprathreshold  contrast  matching  functions  than  at  threshold. 
This  work  has  been  completed  in  the  first  year.  In  fig.  6  are 
low  spatial  frequency  threshold  and  contrast  matching  surfaces 
and  in  fig.  7  are  the  corresponding  surfaces  for  high  spatial 
frequencies.  For  the  threshold  surfaces  the  subject  adjusted 
gratings  on  half  the  display  screen  to  detection  threshold.  For 
the  contrast  matching  surfaces  the  subject  adjusted  the  contrast 
of  the  test  grating  on  half  the  screen  to  match  the  apparent 
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contrast  of  a  standard,  2  c/deg  grating  of  fixed  contrast 
presented  on  the  other  half  screen. 

Even  at  these  moderate  contrasts  (30%)  there  was  very  little 
difference  in  the  physical  contrasts  required  to  match  the 
standard.  As  the  model  predicted,  equal  physical  contrasts 
produced  approximately  equal  apparent  contrasts  over  a  broad 
range  of  spatial  and  temporal  frequencies,  in  spite  of  large 
differences  in  the  retinal  temporal  changes  produced  by  the 
various  patterns. 2 

As  noted  above  (fig.  3)  the  threshold  amplitude  for  drifting 
stabilized  gratings  is  substantially  less  than  the  threshold  for 
counterphase  flickered  gratings.  For  unstabilized  gratings  the 
ratio  of  the  unstabilized  thresholds  depends  on  the  spatial 
frequency  (Watson,  et.  al.,  1980).  The  relationship  between  the 
thresholds  has  been  analyzed  in  terms  of  the  responses  of  medium 
spatial  bandwidth  detector  mechanisms,  and  in  my  previous  work  I 
have  shown  that  apparent  contrast  of  suprathreshold  gratings  is 
also  determined  by  medium  bandwidth  mechanisms.  In  the  first 
year  we  also  examined  the  ratio  of  matching  contrasts  for 
drifting  and  flickering  gratings.  In  fig.  8  are  the  differences 
between  the  mean  log  thresholds  for  drifting  and  flickering 
gratings  and  the  differences  between  the  mean  log  matching 
contrasts  for  several  spatial  frequencies.  As  in  earlier 
experiments  the  threshold  is  much  higher  for  counterphase 
flickered  gratings  than  for  drifting  at  low  spatial  frequencies. 
The  counterphase-flicker  physical  contrasts  matching  the  apparent 
contrast  of  a  drifting  grating  also  had  to  be  higher  at  most 
frequencies,  but  only  slightly.  The  fact  that  stabilization 
eliminated  the  effect  of  spatial  frequency  on  the  threshold  ratio 
(fig.  3)  suggests  that  the  convergence  of  the  threshold  and 
contrast  matching  curves  at  high  spatial  frequencies  may  be  due 
to  eye  movements,  but  no  stabilization  data  have  been  collected 
at  this  point. 

C.  Display  System 

During  the  first  year  a  display  system  consisting  of  a  VAX  11/750 
minicomputer,  an  Adage  3000  image  processor  and  a  Tektronix  690 
RGB  monitor  was  selected,  purchased,  and  installed.  In  the 
course  of  further  investigation  of  the  equipment  originally 
specified  it  became  clear  that  several  changes  were  necessary. 
The  research  plan  required  higher  temporal  and  color  resolution 
than  originally  specified.  The  Adage  3000  actually  purchased  has 
10-bit  digital/video  converters  and  runs  at  60-Hz,  repeat-frame. 

In  discussions  of  my  original  computer  specifications  with 
members  of  Digital  Equipment  Corporation's  display  design  team 
they  became  interested  in  potential  implications  of  the  research 
for  design  of  color  graphics  display  equipment.  In  return  for  my 
giving  two  talks  to  their  design  teams  describing  the  results  of 
experiments  from  my  original  proposal  Digital  provided  a  grant 

2In  the  interim  Bowker  (1983)  has  published  spat iotemporal 
contrast  matching  surfaces  and  noted  the  relative  contrast 
constancy.  He  used  a  less  satisfactory  matching  method  so  his 
data  are  somewhat  less  convincing  due  to  high  variability. 


Figure  6.  Spatiotemporal  threshold  and  suprathreshold  contrast 
sur f aces--Low  spatial  frequencies.  Lower  surface:  mean  log 
threshold  contrasts  for  base-plane  spatial  and  temporal 
frequencies.  Upper  surface:  mean  log  physical  contrast  of 
gratings  matching  apparent  contrast  of  2  c/deg  standard  gratings. 


Figure  7.  Spatiotemporal  threshold  and  suprathreshold  contrast 
surf aces--High  spatial  frequencies.  Lower  surface:  mean  log 
threshold  contrasts  for  base-plane  spatial  and  temporal 
frequencies.  Upper  surface:  mean  log  physical  contrast  of 
gratings  matching  apparent  contrast  of  2  c/deg  standard  gratings. 


Figure  8.  Differences  between  mean  log  matching  contrasts  for 
drifting  and  flickering  gratings.  Open  circles:  subject  LA, 
threshold.  Closed  circles:  threshold  points  from  Watson,  et  al, 
(1980).  Open  squares:  subject  LA,  apparent  contrast  match,  0.7 
C,  2  Hz.  Closed  triangles:  subject  RA,  apparent  contrast  match. 
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of  one-half  the  purchase  price  of  the  VAX  11/750  system. 

While  significant  delivery  delays  have  been  encountered  on 
some  of  the  equipment  ordered  (computer  memory,  disk  packs, 
terminal)  all  major  equipment  is  now  installed  and  running. 
Modifications  of  low-level  software  for  controlling  the  Adage  are 
well  along,  and  cal ibration  of  the  RGB  monitor  should  begin  in 
approximately  one  week. 

Experiments  using  the  system  should  begin  in  about  two 
months,  following  completion  of  programs  to  control  color  output, 
select  and  display  stimuli,  log  data,  etc.  Many  of  the  routines 
are  available  on  the  microcomputer  system  and  need  only  be 
translated  from  BASIC  to  FORTRAN  and  adapted  to  the  new  machine. 
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